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Executive Summary 

State University Of New York 
Report On Performance Indicators 



Scope of ^udlt State University of New York (SUNY) was created in 1948. Today, 

with a total of 64 campuses, SUNY is one of the largest systems of 
public higher education in the nation. These campuses offer academic, 
professional and vocational programs to more than 195,000 students at 
SUNY state-operated/fimded colleges and 196,000 students at community 
colleges throughout the State. The 1995-96 Executive Budget sets 
SUNY’s all-funds operating budget at $3 billion. SUNY provides 
education for almost forty percent of the total higher education population 
of New York. The remainder is provided by the City University of New 
York and a large and diverse set of independent institutions. 

Governmental agencies are accountable for the effective, efficient and 
economical use of publicly ftmded resources provided to accomplish their 
missions. To provide accountability, agency management should 
determine and report measurable results of performance to the public, 
executive management and elected officials to show not only the results 
of current performance, but also the areas where improvements can be 
made. Cornerstones of performance measurement include establishing 
goals pertinent to the agency’s mission and developing performance 
indicators that can measure, by means of inputs, outputs and outcomes, 
the agency’s progress in achieving its goals. 

In 1991, SUNY developed a vision statement entitled SUNY 2000, to 
help establish a plan to take SUNY into the 21st century. SUNY 2000 set 
forth five general goals and 70 specific goals for SUNY. In December 
1994, SUNY’s Office of the Provost issued SUNY’s first Performance 
Indicators Report (Report). The Report’s purpose, in part, was to link 
SUNY’s actual performance to the goals described in SUNY 2000. 

Our audit addressed the following questions about SUNY’s performance 
indicator measurement and reporting for the period January 1, 1993 
through September 29, 1995: 

• Is SUNY’s indicator reporting comparable to that of other 
institutions of higher learning? 

• Do Report indicators reflect input from key Report users and 
measure progress toward SUNY 2000 goals? 

• Are SUNY’s internal controls adequate to ensure the accuracy of 
performance data? 
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Audit Observations 
and Conclusions 


SUNY is the first State agency to undertake the task of developing and 
reporting performance indicators, and has produced its initial Report 
without having been required to do so. We recognize that this undertak- 
ing is an admirable achievement and one that is not without risk. Our 
audit determined that SUNY’s Report could be a more valuable tool for 
management and Report users if SUNY takes the following steps: ensures 
that indicators measure progress toward specific SUNY 2000 goals; 
reports more outcome indicators, as do certain other comparable 
institutions; ensures that indicators reflect the input of key Report users; 
and formalizes its documentation and quality control process to ensure 
the accuracy of indicator data. However, it should be noted that our 
audit did not find any significant misrepresentations in SUNY’s Report. 
We commend SUNY for its production of this first Report and encourage 
its continuing commitment to performance reporting. 

We found that SUNY’s stated criteria for performance measurement are 
very similar to those of the Government Accounting Standards Board 
(GASB). However, we also found SUNY’s Report did not fully satisfy 
all stated criteria; namely, reporting indicators that measure achievement 
of SUNY 2000 goals and that help to stimulate improved performance. 
For example, we found that, while the Report utilized the five generic 
goals stated in SUNY 2000 as a general framework, there was no attempt 
to link the indicators to any of the subset of 70 specific goals presented 
under the five generic goals. (See pp. 6-7) 

Nine of the Report’s 46 indicator sets are output or outcome-related. 
We believe that SUNY should consider using additional outcome 
indicators in its Report. Further, while SUNY did solicit feedback from 
within the SUNY System about how results should be reported, SUNY 
did not involve key external users and could better demonstrate 
accountability to multiple audiences by doing so. (See pp. 7-10) 

Finally, we recommended that SUNY improve its quality control process 
with more formal documentation to enhance the completeness, compara- 
bility, consistency and accuracy of the performance data presented in its 
Report. We believe that implementing such a process would make future 
reports more useful for management, and would serve to strengthen their 
credibility as public presentations of SUNY’s results of operations. (See 
pp. 11-14) 


Comments of 
SUNY Officials 


SUNY officials generally agreed with our recommendations and indicated 
they will incorporate them in the development of future performance 
indicator reports. 
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Response of SUNY Officials to Audit 



Introduction 



Background The state University of New York (SUNY) is one of the largest 

comprehensive systems of public higher education in the nation, with a 
total of 64 campuses. SUNY is comprised of four University Centers 
which offer a full range of academic programs through the doctoral 
level; four Health Science Centers with medical schools; thirteen 
University Colleges which offer comprehensive programs through the 
Master’s level; four Specialized Colleges in Environmental Science and 
Forestry, Maritime studies, Optometric studies, and an upper division 
college of technology; five Statutory Colleges at Cornell and Alfred 
University with specialized offerings in Agriculture and Life Sciences, 
Human Ecology, Industrial and Labor Relations, Veterinary Medicine, 
and Ceramics; six Colleges of Technology and Agriculture which are 
primarily two-year colleges; and thirty Community Colleges which are 
designed to give students job skills or prepare them for transfer to four- 
year colleges. These Community Colleges are supported by State and 
local governments. 

The campuses offer academic, professional and vocational programs to 
more than 195,000 students at SUNY state-operated/funded colleges and 
196,000 students at community colleges throughout the State. A 16- 
member Board of Trustees sets SUNY policy and a SUNY Central 
System Administration unit provides direction and leadership for the 
SUNY system. The 1995-96 Executive Budget contains an all-funds 
operating budget of $3 billion. 

Governmental agencies are accountable for the effective, efficient and 
economical use of pubhcly funded resources provided to accomplish their 
missions. To provide this accountability, agency management should 
determine and report measurable results of performance to the public, 
executive management and elected officials. Through such reporting, 
management and the public can see not only the results of current 
performance, but also the areas where improvements can be made. 
Cornerstones of performance measurement include the establishment of 
goals and objectives that are pertinent to an agency’s mission and 
the development of performance indicators that can measure, by means 
of inputs, outputs and outcomes, the agency’s progress in achieving its 
goals. 

In 1991, SUNY developed a vision statement entitled SUNY 2000, in 
which it set forth five general goals. They are as follows; 



Provide access in undergraduate education. 



Ensure excellence in undergraduate programs and services. 



• Seek, attain, develop, ensure national competitiveness in graduate 
studies and research. 

• Meet State needs in economic development, environmental 
conservation, health care, public education, and social services. 

• Improve management efficiency and effectiveness. 

Historically, SUNY has had more freedom from State regulation than 
other State agencies. In 1986, legislation was passed allowing SUNY 
flexibility in the areas of budget execution, procurement and personnel 
practices. More recently, SUNY has requested greater autonomy for 
its campuses to enable them to directly manage more of their administra- 
tive and financial affairs. SUNY has recognized that, as its receives 
greater flexibihty and autonomy, it needs to be more accountable for the 
resources it manages. 

In December 1994, SUNY issued its first Performance Indicators Report 
(Report). The Report stated that performance indicators are the link for 
coupling autonomy to accountability by presenting tangible evidence of 
results in relation to goals and funding. Performance indicator reporting 
is also a way to present program results in a measurable and meaningful 
maimer. According to the Report’s preface, its purpose was to meet 
increasing demand from the public that higher education report on its 
accomplishments, as well as its problems, in a comprehensive manner by 
focusing on results as well as inputs. The Report was intended to add to 
SUNY’s efforts, initiated by SUNY 2000, to assess, evaluate, and plan 
for the effective and efficient use of its resources. 

In the Report Preface, SUNY noted that it undertook development of its 
performance indicators report without a State mandate requiring it to do 
so. In fact, SUNY is the first State-funded operation in New York to 
have prepared such a report. We recognize that this undertaking is an 
admirable achievement, and one that is not without risks. We encourage 
SUNY to continue its efforts to report the results of performance. We 
beheve, along with SUNY, that performance reporting is a good way to 
ensure system accountability by showing the quality and the quantity of 
programs and services that the campuses provide with the resources 
allocated to them. Furthermore, we hope that SUNY will consider our 
recommendations as areas for potential improvement in its con tinuin g 
efforts to enhance its processes for performance indicator measurement 
and reporting. 
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Audit Scope, 
Objectives and 
Methodology 



We audited SUNY performance indicator measurement and reporting 
processes for the period January 1, 1993 through September 29, 1995. 
The objectives of our performance audit were to determine the following: 
whether SUNY’s performance indicator reporting was comparable to that 
of other institutions of higher education that we contacted; whether 
SUNY obtained input from key Report users in developing its perfor- 
mance indicators; whether Report indicators measure progress toward 
SUNY 2000 goals; and whether SUNY’s internal controls are adequate 
to ensure the accuracy of reported performance data. 

To accomplish our objectives, we interviewed SUNY officials, officials 
from other states involved in performance measuring and members of the 
New York State Legislature. We compared the SUNY Report to 
industry standards detailing the accepted process for developing and 
presenting performance indicator reports. In addition, we examined and 
tested the control procedures that SUNY used to develop the Report, and 
we recomputed selected indicators to test for accuracy and completeness. 

We conducted our audit in accordance with generally accepted govern- 
ment auditing standards. Such standards require that we plan and 
perform our audit to adequately assess those operations of SUNY which 
are included in our audit scope. These standards require that we gain 
an understanding of SUNY’s internal control structure and its compliance 
with those laws, rules and regulations that are relevant to SUNY’s 
operations that are included in our audit scope. An audit includes 
examining, on a test basis, evidence supporting transactions recorded in 
the accounting and operating records and applying such other auditing 
procedures as we consider necessary in the circumstances. An audit also 
includes assessing estimates, judgments and decisions made by manage- 
ment. We believe our audit provides a reasonable basis for our findings, 
conclusions, and recommendations. 

We use a risk-based approach when selecting activities to be audited. 
This approach focuses our audit efforts on those operations that have 
been identified through a preliminary survey as having the greatest 
possibility for needing improvement. Consequently, by design, finite 
audit resources are used to identify where and how improvements can be 
made. Thus, little audit effort is devoted to reviewing operations that 
may be relatively efficient and effective. As a result, our audit reports 
are prepared on an "exception basis." This report therefore, highlights 
those areas needing improvement and does not address activities that may 
be functioning properly. 



Response of SUNY 
Officials to Audit 



A drafy copy of this report was provided to SUNY officials for their 
review and comment. Their comments have been considered in preparing 
this report and are included as Appendix B. 
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Within 90 days after final release of this report, as required by Section 
170 of the Executive Law, the Chancellor of the State University of New 
York shall report to the Governor, the State Comptroller, and the leaders 
of the Legislature and fiscal committees, advising what steps were taken 
to implement the recommendations contained herein, and where 
recommendations were not implemented, the reasons therefor. 





Performance Indicator Reporting 



Performance indicator reporting is globally recognized as an accepted 
method for increasing accountability and for improving the effectiveness 
and efficiency of management. For the past ten to fifteen years, 
Australia, Canada, New Zealand, the United Kingdom and other nations 
have been reporting performance indicators that reflect government 
program results. More recently, efforts have been made at the Federal 
and state levels in the United States to measure the performance of 
government institutions that provide services to the public. In fact, the 
Federal government, as well as a number of states, have now passed 
laws that require that their respective operations to report their perfor- 
mance to the public. For example, the Federal government enacted the 
Government Performance and Results Act of 1993, which requires 
Federal agencies to develop and report program results. 

Eight states have also passed accountability legislation: Florida, 

Colorado, Tennessee, New Mexico, South Carolina, Kentucky, Texas, 
and Wisconsin. Many of these states require that their performance 
indicator reports be audited annually to ensure the reports’ validity and 
credibility. The New York State Comptroller is currently sponsoring 
legislation to create the New York State Government Performance 
Accountability Act. This Act would establish a program whereby State 
agencies would be required to participate in an annual measurement and 
reporting of State government performance, and would be entitled to 
share in any savings from improved productivity. 

Concurrent with the enactment of accountability legislation, a large body 
of criteria has developed that has been widely accepted as necessary to 
produce a valid, useful and accurate performance indicators report. An 
organization that has developed such reporting criteria is the Government 
Accounting Standards Board (GASB), which promulgates standards of 
financial accounting and reporting with respect to activities and 
transactions of state and local governmental entities. In a 1990 research 
paper, GASB recommended that the higher education community use 
selected Service Efforts and Accomplishments (SEA) indicators when 
reporting results to the public. This research paper included a model 
(Model) for measuring performance in public education. The Model is 
based on the premise that publicly funded colleges and universities have 
the responsibility to inform the public about the quality of education 
people are receiving. In April 1994, GASB issued a statement on SEA 
reporting, which provided a general framework of standards for reporting 
SEA indicators by all state and local governmental entities. 
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The GASB Model states that, in order to develop a performance 
indicator accountability system, management must prepare the necessary 
performance indicator information wit hin prescribed guidelines. According 
to the Model, an accountabihty system should include steps that provide 
for identifying potential indicators, linking the indicators to specific 
agency goals, locating data sources and collecting the data. In addition, 
the Model states that an accountabihty system must be credible. Validity 
and credibility are added to performance indicator reporting when an 
independent audit or review is carried out to ascertain the reliability and 
completeness of management’s performance indicator reporting. 



To demonstrate the efficient and effective use of State resources, 
management must define the desired outcomes and set appropriate 
benchmarks or targets to achieve them by identifying the organization’s 
mission and goals. In states where performance reporting is legislatively 
mandated, accountability is defined as a means of judging the effects of 
policies and programs against a set of desired expectations (goals) or 
agreed upon standards. 

SUNY does not beheve that it was bound by the criteria for performance 
reporting set forth in the GASB Model as the process for creating the 
SUNY performance indicator report started well over a year before 
GASB guidelines were published. However, we found that a number of 
public universities in other states (such as Arizona, Florida and 
Tennessee) reported performance, and used five of the eight GASB- 
recommended outcome indicators in doing so. 

SUNY stated that it used four criteria for establishing its performance 
indicators. These criterias are identified in a publication entitled 
“Charting Higher Education Accountabihty: A Sourcebook on State-Level 
Performance Indicators” (Sourcebook). The Sourcebook, published by 
an organization called the Education Commission of the States (Commis- 
sion), is a compilation of performance reporting information contributed 
by the various states, including New York that belong to the Commis- 
sion. Consistent with the Sourcebook, SUNY’s four criteria are: 

• indicators must not diminish or distort the diverse mission of 
constituent campuses; 

• indicators must examine performance on a system-wide basis; 

• indicators must provide tangible evidence of the degree to which 
the goals outlined in SUNY 2000 are being achieved in relation 
to available funding; and 



Indicators’ 
Relationship to 
SUNY’s Goals 
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• indicators must stimulate improved performance in addition to 
describing current results. 

The SUNY criteria for performance indicators are actually very similar 
to those GASB mentioned in 1990 when it recommended the use of SEA 
indicators by institutions of higher education. In analyzing SUNY’s 
Report, we found that it successfully satisfied SUNY’s first two criteria 
for performance indicator reporting, but did not fiilly realize the last two 
criteria. We believe that SUNY could make its report even more useful 
by reporting indicators that provide evidence of achieving the goals 
outlined in SUNY 2000 and that stimulate improved performance. 

For example, we observed that the Report does not mention which 
specific goals each indicator is measuring, and what the eventual targets 
for each goal are. The Report utilized the five generic goals stated in 
SUNY 2000 as a general framework, but there was no attempt to link the 
indicators to any of the subset of 70 specific goals presented under the 
five generic goals. We examined these specific goals in comparison to 
the 46 sets of indicator data presented in the Report. We found that 14 
specific goals appeared to be related to 21 of the indicator data sets. 
However, it was not apparent how the rest of the indicators in the 
Report help Report users measure SUNY’s progress in implementing 
SUNY 2000 goals. 

In addition, SUNY officials told us that the Report is not being fully 
utilized as a tool to help improve performance. While the various data 
sources that make up the current Report were previously used as 
management tools, the Report itself has not served this purpose. We 
believe that SUNY should take advantage of the opportunity to use its 
Report to bring further improvements to the SUNY system. 



Indicators of 

Institutional 

Quality 



When we examined the information from various states as reported in the 
Sourcebook, we identified three models used in the presentation of 
performance indicators: the input/output/outcome model, the quality 

definition model, and the state’s goals (domain) model. All three models 
emphasize the need to demonstrate institutional quality through perfor- 
mance indicator reporting. 

The input/output/outcome model is a common framework used by 
institutions of higher education to categorize and present performance 
indicators. This model assesses institutional quality through input 
indicators (such as number of faculty), output indicators (such as number 
of degrees granted), outcome indicators (such as graduation rates) and 
efficiency indicators (such as student/faculty ratios). Explanatory 
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information (such as funding) is usually provided to help establish a 
framework for the main body of indicators. 

The GASB Model also recommends the use of irput, output and outcome 
indicators to measure an entity’s service efforts and its accomplishment 
of goals and objectives. GASB recommended indicators of outputs and 
outcomes are: 

• results on academic tests; 

• ratings by students and former students; 

• results on professional exams; 

• retention rates; 

• graduation rates; 

• surveys of employer satisfaction with graduates; 

• ratings by students of the adequacy of education programs in 

helping them to reach goals; and 

• rates of employment and/or enrollment in graduate study. 

We found that SUNY followed the uput, output and outcome framework 
to a certain extent. However, our analysis of the Report showed that, of 
the 46 sets of indicator data presented in the Report, only 9 sets relate 
to outcomes and outputs (percent of degrees granted, graduation rates, 
persistence/retention, and student assessment surveys). We believe that 
reporting additional output and outcome indicators may more effectively 
demonstrate results and institutional quality at SUNY. 

A sample of of output and outcome indicators which SUNY’s Report 
does not present, but which are used by public educational institutions 
in some other states and are recommended by the GASB Model, include 
the following: 

• surveys measuring employer satisfaction with graduates; 

• employment rates; 

• pass rates on professional licensing exams; 

• alumni surveys; 
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ERIC 



measurements of students’ perceptions of the quality of instruction 
received before, during and after course work; 



Involvement of Key 
Report Users in 
Developing 
Indicators 



• total student credit hours generated; 

• results of academic tests; and 

• number of degrees awarded by discipline. 

We believe that, by expanding the number of outcome and output 
indicators presented in the Report, SUNY would provide an even more 
detailed picture to the public and the public’s representatives about the 
condition of SUNY and the quality of services it provides. 



The Sourcebook subscribes to the current viewpoint that higher education 
needs to demonstrate its accountability to a broader audience, including 
students, employers, parents and the general public. The study states that 
an indicator that’s useful in demonstrating the quality of an institution 
should reflect the concerns of multiple audiences. 

GASB confirms the Sourcebook’s view that performance indicators 
should involve and be useful for audiences to which information is 
reported. GASB states that, for performance measurement reporting to 
be effective, the opinions and interests of the report’s users must also be 
considered. According to GASB, the views of key report users 
regarding the organization’s goals and how progress toward these goals 
should be measured should be a factor in decisions about which 
indicators to use and how to report them. 

However, we found that SUNY did not contact external Report users to 
determine the kinds of performance information they would find useful 
based on their relationship with SUNY. When we asked some key 
outside users of the Report (including officials from the Legislature and 
the Division of Budget) what they would like to see reported, they stated 
that they were interested in seeing reports of the following indicators: 

• measures of success in the areas of access (SUNY’s availability 
to students who wish to attend), research, grades obtained, jobs 
secured, student and alumni satisfaction and program success (per 
college); 

• SUNY management efficiency and measures of successful 
outcomes, including graduation and job placement rates by 
individual programs; and 



the results of professional exams, employer surveys and compari- 
sons of research revenues to full-time faculty. 



We determined that SUNY officials did request feedback from users 
within the SUNY System. While SUNY did not respond to all the 
feedback, we commend them for their efforts to obtain input from 
University sources. We found that much of the internal feedback (from 
individuals such as SUNY Administration officials, SUNY faculty and 
campus administrators) suggested using the same kinds of output and 
outcome indicators as those mentioned by external users. Internal 
respondents also voiced concern that indicators should be clearly 
explained and set in the proper context. SUNY officials indicate that 
they will be more attentive to the concerns of internal and external users 
in any fumre reports. 



Recommendations 



1 . Consider linking the indicators in fiimre performance indicators 
reports to the specific goals stated in SUNY 2000, and using the 
Report as a management tool to identify opportunities for 
improved performance. 



2. Investigate the feasibility of using additional output and outcome 
indicators to demonstrate the quality of the services provided. 



3. Identify and involve key report users in the development of 
fumre performance indicators reports. 
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Improving Quality Control 



To report meaningful performance data, it is useful for an organization’s 
management to identify appropriate indicators, link the indicators to goals 
and locate and collect indicator data. A quality control system ensures 
that indicators are related to concepts they claim to represent, that data 
sources and collection procedures produce accurate and consistent results 
and that control procedures are adequate to prevent biased reporting. A 
quality control system should also ensure that indicator data is accurate 
and timely, that the process for developing the indicators is well 
documented and that the performance measurement report is independ- 
ently reviewed to enhance its credibility. 

A quality control system enables management to enhance the reliability 
of records by: 

• designing, implementing, and documenting quality control 
procedures; 

• attesting to the accuracy and validity of records at the time they 
are created or updated; 

• retaining any special written program used to extract data from 
a system; and 

• identifying all input and output documents and procedures. 

A complete and accurate documentation of the process that produced 
indicators used in the Report demonstrates the existence and proper 
operation of controls which ensure that records are accurate, reliable, and 
authentic. The documentation should describe the process from beginning 
to end, and detail the meaning, purpose, structure, logical relationships, 
and origins for the indicators. Documentation should also be clear and 
concise so that current and future employees can understand the process. 
Documentation should also be kept up-to-date and available for immediate 
use or review. 

Finally, independent audit and review lends additional value and 
credibihty to performance indicator reports. The role of the independent 
auditor is to ensure that an adequate quality assurance process is in place 
to provide management with the most accurate and useful information. 
Experts in the field of auditing and financial management have reinter- 
preted and modified the American Institute of Certified Public Accoun- 
tants (AICPA) standards for attesting to financial reports to make them 
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applicable to performance measurement reporting. According to these 
standards, an attestation of management’s performance measurement 
reporting would include testing of data. Some of the recommended 
procedures for testing data are listed below. 

• Evaluate management’s performance measurement reporting for 
fairness and balance of presentation. 

• Evaluate management’s performance measurement reporting to 
ascertain that its development is methodologically sound. 

• Evaluate, for reasonableness, the underlying assumptions used by 
management to develop its performance measurement reporting. 

• Ascertain the existence and sufficiency of the documentation 
supporting the performance information reported. Ensure that 
agencies are required to prepare adequate supporting documenta- 
tion and to retain such documentation on file in a maimer suitable 
for review and audit. 

• Evaluate the systems that provided the performance measurement 
reporting data to deter min e if design and controls of the system 
are adequate. 

• Assess the reliability and appropriateness of the performance data. 

• Review management’s performance measurement reporting for 
completeness. 

In accordance with the above recommended processes, we performed a 
detailed examination of 14 judgmentally selected data sets from among 
the 46 data sets that served as the foundation of the indicators used in 
the Report. Although the process SUNY used to establish each indicator 
(including its meaning, origin, purpose, stmcture and logical relationships 
with other indicators) was not fully documented for the 14 data sets we 
reviewed, SUNY was able to recreate the information we needed for our 
review. The data sets we examined were related to the following 
performance areas: peer graduation rates, persistence/retention, graduation 
rates by race and gender, student assessment of educational experiences, 
percent of degrees granted, graduate education and research, average 
number of classes taught, percent of change in full-time equivalent 
faculty and workload, and voluntary support. 



Our examination found exceptions in SUNY’s performance measurement 
and reporting processes related to the completeness, comparability, 
consistency and accuracy of the data reported. However, these findings 
do not represent any significant misrepresentation of SUNY’s position. 
As noted earlier, we also believe that SUNY should improve its quality 
control system with comprehensive documentation of the processes used 
to prepare its Report. We believe that these improvements would enhance 
public presentations of SUNY’s results of operations. 

• Completeness 

SUNY utilized the data available on its automated systems maintained 
within SUNY System offices. Data was also obtained from outside 
sources through manual surveys. SUNY did not disclose in the Report 
the campuses that were consistently omitted from analysis due to their 
non-participation in the automated systems. Further, the Report did not 
indicate those campuses that did not participate in the manual surveys 
conducted by SUNY. Readers should be made aware of the fact that not 
all campuses are included in the SUNY System analysis. 

We also found that SUNY did not disclose those groups of students 
(cohorts) that were used for two indicator data sets, or the methodology 
it used for estimating data in an additional indicator data set. Complete 
information can ensure that readers are able to draw appropriate 
conclusions. 

• Comparability 

Data was available to SUNY which, if included, would have enhanced 
the usefulness and comparabihty of performance for three of the data sets 
presented. We found that SUNY used data from private institutions for 
comparative purposes in some cases (when the comparison enhanced 
SUNY’s position), but omitted such data in other cases (when the 
comparison would not have been a favorable one). A balanced 

presentation of data is necessary to allow readers to fully assess the data 
being presented. 

• Consistency 

We identified instances in which data was used in one indicator data set, 
but was excluded in another even though it appeared that consistency was 
warratned. For another performance indicator data set, we found that 
information was not disclosed about the source of the indicator data or 
the number of campuses that the data represented. In addition, one 
negative indicator was omitted from the Report which, if included, would 
have presented a more balanced picture of SUNY’s performance in this 
area. 
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Accuracy 



We identified eight examples of errors that may have been prevented, 
if, for example, there had been greater redundancy in the development 
and checking of the data summaries and analyses which were used in the 
Report. 

In 1993, the SUNY Board of Trustees resolved to produce the Report 
on an annual basis. We commend SUNY for its co mmitm ent to 
performance reporting. We believe that consideration and implementa- 
tion of the recommendations contained in our report will help SUNY to 
improve the usefulness and credibility of its reporting efforts. 



Recommendation 

4. Improve the quality control process to address the issues raised 
in this report. 
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Appendix A 



STATE UNIVERSITY OF NEW YORK 




OFFICE OF THE SENIOR 
VICE CHANCELLOR 
FOR HNANCEANO 
MANAGEMENT 

System Administration 
State University Plaza 

Albany. NY 12246 JUDC 27, 1996 

518/443-51 ra 
FAX: 518/443-5245 



Mr. Robert H. Attmore 
Deputy Comptroller 
Office of the State Comptroller 
The State Office Building 
Albany, New York 12236 

Dear Bob: 



nOBFr< I H. } 7 y” 
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In accordance with Section 170 of the Executive Law, we are enclosing the comments 
of State University of New York regarding the draft audit report on Performance Indicators, 
State University of New York (95-S-76). 



Sincerely, 



w(] 



/I 
/ / 



■o-' 



/ 



lliam H. Anslow 
Sebior Vice Chancellor 
for Finance and Management 
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State University of New York 
Report on Performance Indicators 
95-S-76 



fieneral C omments 



(SUNY) 



The State University is pleased that the audit of its Performance Indicators Report by 
the Office of the State Comptroller supports this effort of the University. 



We agree with the Office of the State Comptroller that the development and 
publication of a set of Performance Indicators for the University was a worthwhile 
effort and should continue to evolve. We are pleased that the OSC recognizes that 
SUNYs effort was voluntary and that SUNY was the first State agency to do a 
performance report. 

SUNY’s Performance Indicators Report was designed to utilize the considerable 
measures that SUNY has at its disposal without the delay and cost of major new data 
collection efforts. This was seen as a long-range project, and one which would 
expand in future efforts to include constituency groups fi-om whom we did not have 
any information. SUNY’s extensive data base did allow us to generate a long list of 
performance measures. 



Recommendations 



(OSC) 



(SUNY) 



1 . Consider linking the indicators in future performance indicators reports to the 
specific goals stated in SUNY 2000, and using the Report as a management tool 
to identify opportunities for improved p>erformance. 

1 We agree. We did make the linkages within the capabilities of our central data 
bases. Perhaps future efforts should have a campus component which could 
provide for a further linkage to the SUNY 2000 goals. However, any future 
reports will need to consider the new initiatives and directions articulated by 
the SUNY Board of Trustees. 



(OSC) 

(SUNY) 



2. Investigate the feasibility of using additional output and outcome indicators to 
demonstrate the quality of the services provided. 

2. We were committed to getting performance assessment measures from major 
constituency groups such as students, faculty and alunuii. We were able to do 
so for two of these three groups using standard national assessment instruments 
for students and faculty, and in the future, we plan to obtain a comprehensive 
assessment from alumni as well. If possible, this could be extended to 
additional constituencies such as employers as suggested by OSC. 
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(OSC) 

(SUNY) 

(OSC) 

(SUNY) 



3. Identify and involve key report users in the development of future performance 
indicators reports. 

3, We have involved a number of key users in the past and will consider the 
groups mentioned in the audit for future reports. 

4. Improve the quality control process to address the issues raised in this report. 

4. The data bases used to produce the majority of the Performance Indicators have 
been very carefully developed. There are very extensive edits of all data 
collected both on an automated unit record basis and on aggregate survey 
instruments. Summaries and analyses are carefully checked. This is not to say 
that errors do not occur, but the st^ does have an enviable record with respect 
to the quality of its work. For any future reports, we will further improve our 
documentation standards and procedures. 
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NOTICE 

REPRODUCTION BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
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This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket”). 




